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Method of and system for ruiming an algorithm 



TTie invention relates to a method of running an algorithm wherein lie 
algorithm comprises a fizst function and a second fiinction. 

Furthermore title invention relates to a system for running an algorithm 
wherein the algoiithm comprises a first fiinction and a second function. 

5 

An embodiment of the method and the system of the kind set forth above is 
toown from Scalable Video Decoder and its Application to Multi-channel Multicast system 
(IEEE 19th International conference on consumer electronics, year 2000, Page 232 to 233. 
Here, a scalable algorithm is described to be used within real time systems that have limited 

10 resources and time critical algorithms^ An example of a real time system with these 

characteristics is a video decoder to be used within a mnlti-ctLannel multicast system and 
examples of limited resources arc a physical memory, a main processor, and an iupnt/output 
device. The scalable algorithm, for example a decodiag algorithm for video^ comprises a 
plurality of functions like, for example low pass filteiing and upsampling. For each function 

15 the resource requirem^int in terms of CPU load is determined per number of video channels, 
for example 1, 4, or 8 channels, that can be decoded simultaneously. The fianctions that can 
be used within the algorithm are derived from the allowed CPU load and the budget each 
function is allowed to use must be allocated to each fuxiction individually- 

20 It is an object of the cvxrent invention to provide a method as set forth above 

that allocates the resources in an improved way. To achieve this object, the method according 

to the invention comprises the following steps; 

a first step of requesting an algorithm resource by the algorithm to provide a 

plurality of output quality levels, 
25 a second step of determining that the first function provides a first plurality of 

quality levels and the second function provides a second plurality of quality levels^ 

a third step of allocating a budget to the algorithm to enable operating the 

algorithm at a output quality levels said output quality level being one of the plurality of 

output quality levels. 
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a fourth step of assigning a first quality level of tlie first pluraliiy of quality 
levels to the first fanction and of assigning a second quality level of the second plurality of 
quality levels to the second fiinotion. By allocating a budget to aa algorithm as a whole, a 
budget manager, or overall system control does not need to know that the algorithm 
5 comprises of a plurality of functions. The overall system control can therefore be used for the 
gerieral purpose of allocating a budget to algorithms that ron on the system simultaneously. 
The budget is based upon the requested algorithm resources. Instead of allocating a budget to 
the algorithm, the overall system control can set an output quality level to the algorithm as a 
whole. The output quality level can be chosen firom the plurality of output quality levels the 

10 algorithm can provide. Each fbnction of the algorithm can provide a plurality of qxzality 

levels. When an algorithm gets allocated a bxidgct or is assigned an output quality^ a quality 
control can assign a corresponding quality level or setting to each function. The 
corresponding quality level or setting is chosen firom the plurality of quality levels that can be 
provided by a function. The quality level of the function that provides the highest output 

15 quality level of the algorithm for the allocated budget can be the preferred choice &om the 
plurality of quality levels that can be provided by a fimctiojx The quality control distributes 
implicitly its allocated budget over the functions the algorithm comprises by the assigximent 
of a corresponding qualiiy level or setting to each function. The assigned quality level per 
function is based upon the budget that is allocated to the algoritibm. 

20 An embodiment of the method according to the invention is described in claim 

2. A function can provide a quality level for a plurality of levels of complexity wherein a 
level of complexity for example is determined by a number of mathematical operations a 
function can perfona, an amount of memory the function requires or communication means, 
like bandwidth, the function requires ► When the algorithm consists of a plurality of functions^ 

25 each ftmotion providing a plurality of quality levels. There are a lot of combinations of level 
of complexity and quality level possible. A quality control can perform these combinations 
and can decide upon these combinations which quality level to assign to a function. 
Furthermore* the knowledge about the complexity of a fianction while providing the same 
quality level can lead to more smooth output quality transitions of The algorithm as a whole. 

30 An embodiment of the method according to the invention is described in claim 

3. Each function can operate at its own quality leveL A combination of the first function and 
the second function can lead to one algorithm that can provide a plurality of output quality 
Jjeveis. When a new budget is allocated to an algorithm which leads to a different output 
quality level, the same algorithm can be operated again, by allocating new quality levels to 
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the first and second function as previously described. A imtnber of algorithms providing a 
same fianctionality but at a different output quality level that can, for example^ be operated in 
parallel, can be limited this way* 

An embodiment of the method according to the invention is described in claim 
5 4, A quality control^, for example can choose for each function the lowest complexity for die 
highest quality level from the plurality of combinations of complexity and quality level per 
flincTion, 

An embodiment of The method according to the invention is described in claim 

5, When the allocated budget is substantially equal to the requested algorithm resource, the 
10 algorithm docs not get allocated substantially more than, its requested algorithm resowce, 

This prevents resources being not used by the algorithm, which can cause rejection of other 
algorilimis to operate because their requested resource is already allocated to ttie algorithm. 

An embodiment of the method according to the invention is described in claim 

6. When the firsi amount of resources in addition to the second amount of resources is 

15 substantially equal to the allocated budget, the algorithm docs not use substantially more than 
its allocated budget. This prevents budget-overrun by the algorithm which may cause budget 
shortage of other algorithms or algorithms that can result in missing deadlines of these other 
algorithms or algorithms and degradation of an overall output quality. 

An embodiment of the method according to the invention is described in claim 

20 7, The output quality levels thai can be provided by an algorithm can depend upon a 

hardware platform that the algorithm is operated upon. Whea for example, the fibrst fimction 
of the algorithm has specific hardware requirements, like for example the availability of a 
harddisk, the first fimction may be omitted when the hardware is not available. 

An embodiment of the method accorcfing to the invention is described in claim 

25 S. The output quality levels that can be provided by an algorithm can depend upon a software 
platfomi that the algorithm can access. When, for example, Hie first function of the algorithm 
has specific softwiare platform requirements, like for example the availability of a linear 
interpolation algorithm, the first function may be operated differently when the linear 
interpolation algorithm is not available^ for example by using an available cubic interpolation 

30 algorithm. 

A further object of the invention is to provide in a system as set forth above 
that allocates the resources in an icnproved way. To achieve this object, the system according 
to the invention comprises: function means conceived to contain the first function of the 
algorithm and a second fiinctiou of the algorithm, 
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lookup maang conceived to contain a plurality of output quality levels that can 

be provided by the algorithm, a first plurality of quali-^ level settings of the fJxst function and 

a second pli^rality of quality level settings of the second function. 

Embodiments of the system according to the invention are described in claims 

5 10tol3» 



The invention will be described by means of embodiments shown by the following drawings: 
Figure 1 illustrates an embodiment of the main steps of the method according 
TO the invention, 

1 0 Figure 2 illustrates an example of an algorithm that comprises a plurality of 

functions. 

Figure 3 illustrates an example of a diagram in which the complexity of the 
algorithm is set against the output q^uaiity level that can be provided by the algorithm. 

Figure 4 iUusirates the most important parts of an embodiment of the system 
1 5 according to the invention in a schematic way^ 

Figure 5 illnstratcs a storage device in a schematic way that comprises an 
embodiment of a storage device comprising a computer program product arranged to perform 

the method according to the invention, 

Figure 6 illustrates a television set in a schematic way that contains an 
20 erabodlxnent of the system according to the inventian. 

Figure 7 illustrates a set-top box in a schematic way that contains an 
embodiment of the system according to the invention. 



25 Figure 1 illustrates an, embodiment of the njain steps of the method according 

to the inventiorL Programmable components, rather than dedicated smgle-fimction 
components can perform continuous media processing. Those single-fimction components 
are used in traditional television receivers in which some of those single-fhnction 
components could be combined to perform for example color decoding for NTSC or PAL 

30 systems, noise reduction or frame rate up-conversion. With the introduction of programmable 
componjen.Tsip contiimous media processing algorithms can be implemented in software 
instead o£ hardware. Some of the expected advantages of the software implementation of 
media processius algorithms are: reduced tune to market, re-use of hardware, re-use of 
software algorithms, portability^ and flexibility- The software implementation of the media 
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processing algorithms must run Within the real-tune ezivironment ia which system resources 
are finite and sufiScient system resources inay not he reserved for a particular procassing 
algoritbm, which can lead to changes in the output quality provided by the particular 
processing algorithm. Output qt^ity levels can be measured by perception laeasurements^ or 
5 objectively by available measurement means, A system ruiming the processing algorilhms is 
able to provide hi^-quality audio and video that has a relatively higji firam^ rate above 50 
HZi, aknost no tolerance for frame rate fluctuations and a low tolerance for irame skips. 
Preferably, the system is also able to provids low frame rates With a maximum of 30 Hz, a 
high tolerance for ftame rate fluctuatiDns and a high tolerance for frame skips. 
1 0 Algorithms can be allocsted budget explicitly or implicitly by setting an output 

quality level of the algorithm. One of the objectives of an overall system control is to 

optiinize the total output quality provided by the total system while making efficient use of 
all the available resources. The total output quality provided by the system depends amongst 
others upon the number of algorithms operating concurrently and the data an algorithm 

15 processes. The system may "be^ for example^ a television, a PC, a display, a set-top box, or a 
VCR. In order to achieve this objective, the main steps below are performed. 

Here, step 100 is an initialization step in which an overall system contcoL for 
example a budgcrt manager accesses the contents of a first lookup table as iUustrated in Table 
1. ha this table, '^CPU'^ "co-processor*' and "memory requirements" are examples of 

20 resources that an algorithm can use. Furthmnore, a higher number that is denoted in the 
column named "quality number" indicates a better output quality level perceived by a user. 
By accessing the contents of this first lookup table^ The overall system control detennines the 
predefined amount of resources^ for example CPU cycles, an algorithm requests to provide a 
predefined output quality level. 

25 



Quality number 


CPU 


co-processor 


memory requiremeats 




[cycles] 


[cycles] 


[bytes] 


79 


39 


22 


3 


68 


28 


12 


3 



Table 1 



The algorithm, or a quality control, which is pare of the algorithm, can have access to the 
30 contents of Table 1 too. Algorithms are started implicitly when a user svvitohes to another 
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channel for a naain window and tlie, analog, source of the new channel is different firom the, 
digital, soiurce of the old channel. Other examples of starting algorithms or changing the 
resource requirement of an algorithm are: 

when a user exchanges the contents of a main window and a picture in picture 
5 window by for example viewing tlie re-play after a goal, 

when the size of a video conferencing window changes^ or 
whea a new application, for example a video conferencing application when a 
call airivesj is started in an additional window. The resource requirement of an algorithm 
changes too when Th.e media data the algorithm processes changes. A change in die media 

1 0 data can be caused by the service provider that may transmit sources with different input 
parameters, for example when a movie which can be a 24 Hz film is interrupted by a 
commercial which can be a 60 Hz camera or it can be caused by motion or scene changes, 

WiiMn step 102 the functions a media processing algorithm comprises is 
determined by, for example^ reading this infomiation fiom some configuration file. This file 

15 describes that, for example, an algorithm for edge or shaipness enhancement comprises the 
functionfl as is illustrated in a $cheraatic way In Figure 2. Wifbin this figure, a detail filter 
2QQ, a non-linear function 202, a gain 204, an adder 206;, and noise measurement 208, is 
shown. The detail filter extracts higher fiequency components ftom an input signal 
containing a video signal. Those components can be added to the input signal to increase the 

20 overall sharpness impression of the video signal. The non-linjsar function and following gain 
can reduce artifacts like clipping caused by the detail filtsr whereas the noise measurement 
function can adapt the sharpness enhancement dependent upon the IlOise level contained 
within the input signal. 

Within step 104 the requested resources and the quality level per fimction are 

25 detenniaed. In order to separate concems, a quality control 2103 see Figure 2^ shields the 

overall system control from the functions an algorithm comprises and is part of the algorithm. 
Within the algorithm for edge or sharpness enhancement, the detail filter varies its requested 
resources for, for example, number of CPU cycles or number of bytes. The variation of 
requested resources is determined by the quality levels described by the coefficients for the 

30 filter and the type of filter; horizontal, vertical j or boHi, The non-linear function varies its 
requested icsources for, for example, CI*U cycles and is detenniaed by the quality levels 
described by the quantization of the non-linear fimction, which can differ for the input signal 
and output signal. The gain varies its requested resources for, for example, a multiplier, shifi; 
and adder operations dependent upon whether it is stored as fixed values in a lookup table 
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contained in memory, it is calculated by a multiplier or is calculated by shift and adder 
operations. The noise measurement varies its requested resources for, for example CPU 
cycles, because it can, for example, be switched on or off. The adder operation can vary its 
requested amount of resources by for example doing less precise additions. However, adder 
5 operations that caonot vary theix requested amount of resources but provide a predefined 
quality level for a predefined amount of resources, for example CPU cycles, can be used too. 
The combination of all settings for requested resources and quality level per functionp results 
in a large design space in which tiie complexity of the algorithm, or of a function of the 
algorithm, or of a combination of functions of the algorithm is set against its quality level. 

10 The result is summarized into, for example, a second lookup table as illustrated in Table 2. In 
this table, there are three main columns: "version", which assigns a unique number to a row, 
"quality", which groups all parameters concerning the output quality level a algorithm can 
provide, and "complexity", which groups all parameters concerning the complexity of the 
algorithm. The mentioned parameters are not limiting, for example^ store operations or 

15 communication means like bandwidth and cache can be used as parameters concerning the 
complexity of the algorithm. Furthermore, the numbers used for the quality and complexity 
within Table 2 are absolute, but may be normalized to operations per pixel within video 
independently from the chosen format. With this concept, a media algorithm designer designs 
the functions the media algorithm comprises to provide the correct functionality at different 

20 output quality levels. 
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Table 2 



Wittiiii Step 106 the contents of the table is updated for tiie available software 
5 platform the algorithm has access TO. For example, when the software platform does not 
support ''temporal processing'', this column is removed ficom the table and the effecTliaied 
rows arc updated coircspondingly. Wlien for ejcample "horizontal processing" is not 
supported, the corregponding colximn and the rows which do uot lead to any processing at all, 
like row 6 and N, arc removed from xhe table. It is also possible to instantiate a run-time 
1 0 lookup table contaiiung a mapping from software functions available within the software 
platforai to software functions required by the algorithm instead of updating Table 2, 

Within step 108 the contents of the table is updated for the available hardware 
platform the algorithm must be operated upon. For example^ when the hardware platforai 
does uot provide a co-processor, this column is removed from the tabic and all rows are 
15 removed that only used a co-processor. It is also possible to instantiate a run-time lookup 
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table containing a mapping from hardware available wi&in the hardware platform to 
hardware required by the algoiitfim instead of updatiixg Table 2. 

After these steps, the functions an algorithm comprises, the plurality of qxjality 
levels the difierent functions provide, the plurality of output quality levels the algorithm 
5 provides and the hardware and software the algorithm requires firom the hardware platfomi 
and software are known to the quality control. The overall system control only needs to know 
about the algoritibctn, the resotirces the algorithm requests^ the hardware it requires and the 
plurality of output quality levels the algorithm provides. 

Within step 1 1 0 the overall system control allocates a resource budget to the 

10 algorithm in accordance with a best overall system's output quality level- A best overall 

system's output quality level can be achieved when the system is in a steady state in which all 
algoriflims that are ruoning provide a predefined output quality level and the system is fiilly 
loaded- This means that additional algorithms can not be started witliout adjusting the output 
quality levels of the running algorithms. The budget thai is allocated is substantially equal to 

15 the resources reqiiested by the algorithm to provide a predefined output quality level. When 
the algorithm gets allocated less budget than requested^ the algorithm may not provide the 
predefined output quality level and when the algorithm gets allocated more budget than 
requested, the algorithm may not use all resources. The overall system control 212, see 
Figure 2, allocates the budget to the algorithm based upon the contents of Table 1 . The 

20 overall system control 212 can aHocate "dhe budget to the algorithm based upon the contents 
of Table 2- In the latter case, the overall system control decides upon a more smooth 
transition of the output quality level provided by the algorithm. As is Shown in Table 2, an 
abmpt transition of the output quality level provided by the algorithm is likely from version 2 
to version 6, or from version 5 to version 6, because the processing changes iu two 

25 dimensions- A more smooth transition is expected from version 2 to version 3 , because the 
processing changes only in one dimension. The other "quality" parameters lilce the quality 
number or PSNR also provide information about ih.© smoothness of transitions. The 
information about the hardware platform and software platform as derived in steps 106 and 
lOS, can also be accessed by the overall system control. Selection of the output quality level 

30 provided by iiie algorithm is then based upon for example the available hardware. This is 
shown in Table 2, where version 1 and version 2 provide the same output quality level 
because their quality numbers are equal, but they distribute the required resources differently 
between the CPU and co-processors. The quality control can use this knowledge of 
distribution between CPU and co-processors to deal, amongst others, at run-time with 
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overload situations in which the qtiality control can change the distxibunon, while the 
algorithm still provides the same outpm qi^ality. 

Within step 112 the quality control translates the allocated bttdget, or output 
quality level, to the algorithm as a whole mto a quality level allocation to the different 
5 functions the algorixhm comprises. This translation is based upon the contents of Table 2 and 
takes the oombiaa-tion of all settings for requested resources and quality level per function 
iato account The combination of all settings for requested resources determines the 
complexity of the algoritimi^ a function of the algorithm and a combination of llie functions 
of The algorithm. The cpmplexity is expressed with a number. This number is weighted to get 

10 a single number for a specific hardware or software platform. Figure 3 shows an example of 
the combination of the complesdty and provided output quality level. Useful combinations 
are achieved at the highest output quality for the lo^vest complesdty as is uidicated by the 
draivn curve in Figure 3. Each dot implies different quality settings or quality levek for the 
functions an algorithm comprises. The information about Hit hardware platform and software 

15 platform as derived in steps 106 and 108 is accessed by the quality control. The quality 

control uses this information to choose the best oombisution of the oomple>£ity and provided 
output quallly level, because the best combination and the number of combinations can 
depend upon the hardware and/or software platform the functions must run upon. Changing 
the tesolution of video data by for example sub-sampling the video data or deleting entire 

20 frames, liaos or pixels is prevented by inierpretation Of the contents of Table 1 and Table 2 as 
previously described. The quality control also maximizes the perceptual quality because a 
user perceives a low output quality provided by the system, when the quality of, for example 
a movie, is changed continuously. Therefoxe quality levels are sparingly adjusted. 

Within step 114, the quality control re-allocates the translated allocated budget 

25 to the functions the algorithm comprises implicitly, by assigning the corresponding quality 
level to the functions- 

Within step 1 16 the functions and therefore the algorithm as a whole start 
operating using their allocated budget and set quality level. After completicn of the 
algorithm, step 100 can be performed again or the final step 11 8 is reached. 

30 The order in the described embodiment of the method of 1±ie current invention 

is not mandatory, a person sicilled in the art may change the order of steps or perform steps 
concurrently using tiareading models, multi-processor systems or multiple processes without 
departing from the concept as intended by the current invention, FuithcraLoxe, the introduced 
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quality control and overall system control express roles or concepts tiiat can be used witibln 
the method of T5ie current invention. 

Figure 4 illustrates the most important parts of an embodiment of the system 
according to the invention in a schematic way. The system, 400, comprises a firjst memory 
5 402 tiiat contains per resource an amount of that resource an algorithm requests to provide a 
predejSned otrtput q^ity level. A CPU and a co-processor are examples of resources of 
which cycles can be requested, A second memory 404, contains a module to pcsrfbrm a first 
ftmcticn of the algorilJimj while a third memory 406, contains a module to perfonn a second 
function of the algorithm. Consider the edge ox sharpness enhancement algoritibm 214, as 

10 described in Figure 2, The second memory 404 contains the module that parfoims detail 
filtering 200, while the third memory 406 contains the module that performs noise 
measurement 208. The system may also contain more memories cozxtaining modules that 
pexfoim all the functions of the edge or sharpness enhancement algorithm as described in 
Figure 2. The fourth memoiy^ 408 contains a lookup tabic containing per equality level of a 

1 5 first plurality of quality levels the first fonction can provid&p the amount of resources it 
requires. The fifth memory^ 410 contains a lookup table containing per quaHty level of a 
second plurality of qualiiy levels the second fiandion can provide, the amount of resources 
the second fimction requires, After the overall system control allocates a budget per resource, 
as previously described, the siscth memory 412 contains per resource the amount of budget 

20 allocated to the algorithm. The overall system control ean also assign the output quality level 
of the algorithm as a whole^ thereby implicitly allocating a budget per resource to the 
algorithm- Furthermore, the memories 414 and 4l6 contain the quality levels provided by the 
first and second function of the algorithm respectively. Memory 418 contains a plurality of 
complexity numbers indicating a plurality of levels of complexity of op^ation of the first 

25 function of the algorithm. Memory 420 contains a complexity ntimber indicating the least 
complex level of operation of the first fianction of the algorithm. La order to determine the 
content of memories 414 and 416 as previously described the quality control has access to the 
contents of memory 418 and 420. The contents of memories 402, 408, and 41 0, 41 8, 420 can 
also be combined into one lookup table as illustrated in Table 2. This combined lookup table 

30 can then be stored into one memory instead of more separate memories, FuTfliermore, when 
the system is realised in silicon in which the functions and lookup table are hard-wired to 
eactioTher, the memories 412, 414 and 416 may be omitted. The quality control accesses the 
contents of memory 426 that contains a conflguraxion file containing information about the 
available hardware within the system and it accesses the contents of memory 428 that 
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contciins a configuration file conraining information about lie available software algorithms 
Wttiin the system. The quality coutrol has access to all previously described memorieSa 
whereas the overall system control only requires access to memories 402, 412, 42(S and 428. 
Howover, when the overall system control has access to more membriesj the output quality 
5 level transitions provided by the algorithm can become smoother The system also comprises 
a first co-processor, 422^ on which the first function of the algorithm can nm and a second 
co-processor, 424, wMch on which the second function of the algorithm can run. An, 
optionalj CPU^ 430, operates the algorithm as a whole^ because there needs to be some inter- 
process communication between the first and second function. When the system does not 

10 contain co-processors, fhc functions and therefore, the algorithm run on the CPU. When the 
first function can be operated a* a plurality of levels of complexity, the first function runs at 
the least complex level on a dedicated co-processor 432, while the more complex level is run 
on tlie co-processor 422- It is also possible that the first fimction runs at each of the plurality 
of levels of complexity on either 432 or 422 or that ihe firsi fimction runs at each of the 

15 plurality of levels of complexity on CPU 430. This system can be realised in software 

intended to be operated as an application by a computer or any other standard architecture 
able to operate software. The system can be used to operate a digital television set 434* The 
system can also be realised in silicon wherein ttte mentioned loolcup tables are replaced by 
logical building blocks that are hard-wired to each other and the mentioned processors and 

20 co-processors are omitted. 

Figure 5 illustrates^ in a schematic way, a storage device that comprises a 
computer program product ananged to perform the method according to tile invention. Here, 
500 is a compact-disk comprising code 502. 

Figure 6 illustrates, in a schematic way, the most important parts of a 

25 television set that comprises an embodiment of the system according to the invention. Here 
an antenna, 600 receives a television signal. The antenna may als;o be for example a satellite 
dish, cable, storage device, internet^ Ethernet or any other device able to receive a television 
signal. A receiver^ 602 receives the signal. The signal may be for example digital^ analog, 
RGB or YUV. Besides the receiver 602, the television set contains a piogrammablc 

30 component, 604, for example a programmable integrated circuit, This programmable 

component contains a system according to the invention 606. A television screen 608 shows 
images that are received by the receiver 602 md arc processed by tbe programmable 
component 604. the system according to the invention 606 aad other parts that are normally 
contained in a television set, but are not shown here. 
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Figure 7 illustcates, in a schematic way, the most importaat parts of a set-top 
box thai comprises an embodiment of the system according to the invention. Here, an antenna 
700 receives a television signal. The antenna may also be for example a satellite dish, cable^ 
storage device, internet, Ethernet or any other device able to receive a television signal. A 
5 set-top box 702, receives the signal. The signal may be for example digital, analogue, RGD 
or YUV. Besides the usual parts that are contained in a set-top box, but arc not shown here, 
the set-top hoK contains a system according xo the invention 704, The television set 706 can 
show the output signal generated from a received signal by the set-top box 702 together with 
the system according to the invention 704. The output signal may also be directed to a 
1 0 storage device like a VCR^ DVD-RW or a harddisk or they may be directed to an internet 
link in stead of being directed to the television set. 



